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ABSTRACT  The  use  of  computer-based,  psychomotor  testing  systems  for  personnel  selection  and  classification  has 
gained  popularity  in  the  civilian  and  military  worlds  in  recent  years.  However,  several  issues  need  to  be  resolved  before 
adopting  a  computerized,  psychomotor  test.  The  purpose  of  this  study  was  to  compare  the  impact  of  alternative  input 
devices  used  for  the  Test  Of  Basic  Aviation  Skills  (TBAS)  as  well  as  to  explore  the  practice  effects  of  the  TBAS.  In  study 
1,  participants  were  administered  the  TBAS  tracking  tests  once  with  a  throttle  and  once  with  foot  pedals  in  a  classic  test- 
retest  paradigm.  The  results  confirmed  that  neither  of  the  input  devices  provided  a  significant  advantage  on  TBAS  perfor¬ 
mance.  In  study  2,  participants  were  administered  the  TBAS  twice  with  a  24-hour  interval  between  testing.  The  results 
demonstrated  significant  practice  effects  for  all  the  TBAS  subtests  except  for  the  dichotic  listening  tests. 


INTRODUCTION 

Presently,  the  U.S.  Navy  is  engaged  in  the  revalidation  and 
redevelopment  of  the  Aviation  Selection  Test  Battery  (ASTB). 
The  ASTB  is  a  test  of  basic  aviation  skills  and  abilities  admin¬ 
istered  by  the  Naval  Operational  Medicine  Institute  (NOMI). 
It  is  the  primary  basis  for  selection  of  potential  candidates  for 
Navy  and  Marine  Corps  aviator  training  programs.  The  ASTB 
is  primarily  administered  using  a  paper-and-pencil  format  but 
an  equivalent  Web-based  format  is  also  available.  As  part  of 
an  effort  to  improve  aviation  selection  tests,  the  U.S.  Navy 
in  collaboration  with  the  U.S.  Air  Force,  is  evaluating  other 
selection  tools  such  as  the  performance-based  Test  of  Basic 
Aviation  Skills  (TBAS).  However,  before  incorporating  a 
performance-based  test,  careful  attention  must  be  given  to 
issues  such  as  the  effects  of  alternative  input  devices  and  mul¬ 
tiple  test  exposures.  The  current  studies  were  designed  to 
examine  these  issues  related  to  the  TBAS. 

From  the  early  days  of  military  pilot  selection,  apparatus- 
based,  psychomotor  testing  has  been  used  in  the  military  avi¬ 
ation  personnel  selection  process.  Although  psychomotor 
testing  has  evolved  significantly  with  historical  events  and 
technological  advances,  it  has  remained  as  one  of  the  principal 
categories  of  the  military  pilot  selection  batteries.'  In  recent 
years,  computer-based  tests  have  replaced  the  traditional 
apparatus  tests  for  the  measurement  of  psychomotor  ability. 
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In  1981,  the  U.S.  Air  Force  initiated  a  project  to  develop 
a  computer-administered  test  battery  for  pilot  selection  and 
classification  based  on  studies  that  demonstrated  the  reliabil¬ 
ity  and  validity  of  psychomotor  testing. The  resulting  prod¬ 
uct  was  the  Basic  Attributes  Test  (BAT).  The  BAT  is  designed 
to  assess  psychomotor  and  cognitive  skills  and  abilities,  as 
well  as  risk-taking  attitudes.  It  consists  of  five  subtests  that 
include  two-hand  coordination,  complex  coordination,  mental 
rotation,  item  recognition,  time  sharing  and  activities  inter¬ 
est  inventory.  The  BAT  administration  requires  a  386-based 
computer,  2  joysticks,  and  a  number  pad.  Candidates  who  per¬ 
form  well  on  the  BAT  are  found  to  be  more  likely  to  com¬ 
plete  ground  school,  require  fewer  hours  of  training,  have 
higher  class  rankings,  and  complete  advanced  jet  training."*  In 
1999,  Air  Force  Training  Command  requested  development 
of  a  new  skills  battery  for  pilot  selection  that  uses  the  latest 
advances  in  psychomotor  and  cognitive  research.^  As  a  result 
of  this  effort,  the  TBAS  was  developed  as  an  improved  ver¬ 
sion  of  the  BAT.  The  TBAS  has  been  operational  since  2006. 
The  TBAS  consists  of  a  series  of  individual  tasks  such  as  the 
directional  orientation  test,  3-digit  listening  test,  5-digit  listen¬ 
ing  test,  horizontal  tracking  test  (HTT),  airplane  tracking  test, 
and  emergency  scenario  test  as  well  as  a  series  of  multitasks 
that  are  combinations  of  two  or  more  of  these  tests  admin¬ 
istered  simultaneously.  These  tests  are  designed  to  measure 
spatial  apperception,  selective  attention,  coordination,  and 
decision-making  abilities  under  stress.  The  TBAS  administra¬ 
tion  requires  a  specialized  computer  system  that  includes  joy¬ 
stick,  throttle  or  rudder  pedals,  headphones,  and  keypad. 

Unlike  the  Air  Force,  the  U.S.  Navy  relies  primarily  on  the 
ASTB  to  select  candidates  for  its  aviation  pilot  and  flight  officer 
training  programs.  The  latest  version  of  the  ASTB  was  devel¬ 
oped  by  the  NOMI  in  cooperation  with  Educational  Testing 
Services  (ETS)  in  Princeton,  New  Jersey  and  was  imple¬ 
mented  in  1993.®  The  ASTB  is  designed  to  assess  aviation- 
specific  skills  and  abilities  that  are  predictive  of  primary  flight 
training  success.^  It  consists  of  4  subtests:  math  and  verbal 
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test  (MVT),  mechanical  comprehension  test  (MCT),  aviation 
and  nautical  information  (ANI)  test,  and  spatial  apperception 
test  (SAT).  Each  of  these  subtests  measures  specific  abilities 
(e.g.,  spatial  apperception  subtest  measures  spatial  ability). 
Weighted  combinations  of  these  subtest  scores  are  compiled 
to  compute  composite  scores.  Academic  qualification  ratings 
(AQR),  pilot  flight  aptitude  rating  (PEAR),  and  flight  officer 
flight  aptitude  rating  (FOFAR)  of  the  ASTB  are  the  compos¬ 
ite  scores  relevant  for  flight  training.  The  AQR  has  been  vali¬ 
dated  to  predict  academic  performance  in  ground  school  and 
the  PFAR/FOFAR  has  been  validated  to  predict  primary  flight 
training  grades.^ 

Because  the  ASTB  primarily  measures  basic  cognitive  skills 
and  abilities,  it  may  not  be  an  adequate  measure  of  the  com¬ 
plete  set  of  cognitive  and  coordination  aptitudes  related  to  suc¬ 
cessful  flight  training.  Valid,  comprehensive  computer-based 
tests,  on  the  other  hand,  can  enhance  the  prediction  of  flight¬ 
training  success.*  Therefore,  in  recent  years,  the  Navy  has  been 
involved  in  revalidation  and  modification  of  the  current  avia¬ 
tion  selection  process  and  has  been  planning  to  incorporate  a 
performance-based  psychomotor  test  such  as  the  TBAS. 

In  an  effort  to  incorporate  the  TBAS,  the  initial  step  was 
to  select  the  particular  input  devices  that  are  appropriate  for 
naval  aviators.  The  purpose  of  incorporating  devices  such  as 
the  foot  pedals  and  joysticks  is  to  measure  abilities  that  are 
relevant  in  flying  an  aircraft.  In  an  aircraft  the  rudder  pedals 
are  operated  with  the  feet.  The  right  rudder  pedal  is  used  to 
make  the  aircraft  yaw  (i.e.,  rotate  about  the  vertical  axis)  to 
the  right,  and  the  left  pedal  is  used  to  make  it  yaw  to  the  left. 
The  primary  purpose  of  the  rudder  is  not  to  change  direction 
of  the  aircraft,  but  mainly  to  balance  it  in  turns,  or  to  compen¬ 
sate  for  winds  and  the  effects  from  other  controls.’  In  a  fixed- 
wing  aircraft,  although  the  rudder  is  used  during  takeoffs  and 
landings,  it  is  not  typically  used  in  flight  when  the  aircraft  is 
properly  trimmed. 

When  flying  a  fixed-wing  aircraft,  a  pilot  primarily  uses 
the  hands  to  operate  the  control  stick  or  yoke  and  the  throttle. 
The  control  stick  or  yoke  is  used  to  control  the  direction  and 
orientation  of  the  aircraft  relative  to  the  horizon  (i.e.,  the  atti¬ 
tude  of  the  aircraft),  in  both  pitch  and  roll.  The  throttle,  on 
the  other  hand,  is  used  to  control  the  engine  speed.’  A  pilot 
often  needs  to  use  both  hands  to  have  simultaneous  access  to 
the  control  stick  or  yoke  and  the  throttle.  Thus,  coordination 
between  the  hands  seems  to  be  particularly  important  when 
flying  a  fixed-wing  aircraft.  Moreover,  it  is  easier  to  standard¬ 
ize  a  throttle  rather  than  foot  pedals  as  an  input  device  across 
various  testing  facilities.  Interestingly,  the  original  BAT  actu¬ 
ally  uses  two  joysticks  and  does  not  involve  rudder  pedals 
but  the  newly  developed  TBAS  uses  both  joystick  and  rudder 
pedals.  The  next  revision  of  the  ASTB  plans  to  incorporate  a 
modified  version  of  the  TBAS  that  replaces  the  foot  pedals 
with  a  throttle.  However,  input  devices  can  affect  the  rate  and 
ease  of  response  in  a  computer-based  test.'°  Therefore,  it  is 
imperative  to  evaluate  the  impact  of  alternative  input  devices 
on  TBAS  performance  before  making  such  substitution. 


Study  1 

The  objective  of  this  study  was  to  examine  the  impact  of  sub¬ 
stituting  a  throttle  control  for  foot  pedals  during  the  TBAS 
horizontal  tracking  tests.  The  goal  was  to  determine  whether 
such  substitution  can  affect  TBAS  performance. 

Method 

Participants 

Students  waiting  to  attend  aviation  preflight  indoctrination 
(API)  were  recruited  to  take  the  TBAS.  A  total  of  47  students 
participated  in  the  study.  However,  TBAS  records  for  4  stu¬ 
dents  were  not  complete  and  a  final  set  of  43  TBAS  scores 
were  used  for  the  analysis.  Out  of  the  43  participants,  39  were 
male  and  4  were  female.  The  sample  of  participants  included 
38  Caucasians,  2  African  Americans,  1  Asian,  1  Hispanic,  and 
I  Native  American. 

Materials 

TEAS 

The  TBAS  is  a  revised  version  of  the  BAT.  It  is  a  comput¬ 
erized  test,  which  measures  psychomotor  coordination  and 
information  processing  speed."  This  test  takes  approximately 
1.5  hours  to  administer.  The  TBAS  system  is  compatible 
with  Windows  XP  operating  system  and  requires  a  monitor,  a 
Thrustmaster  HOTAS  Cougar  USB  joystick  and  throttle,  CH 
Products  Pro  pedals  USB  flight  simulator  pedals,  a  bidirec¬ 
tional  headphone,  and  a  keypad. 

The  TBAS  consists  of  a  series  of  individual  tasks  such  as 
the  directional  orientation  test,  3-digit  listening  test,  5-digit 
listening  test,  horizontal  tracking  test,  airplane  tracking  test, 
and  emergency  scenario  test  as  well  as  a  series  of  multitasks 
which  are  combinations  of  two  or  more  of  these  tests  admin¬ 
istered  simultaneously. 

The  directional  orientation  test  measures  spatial  orientation 
abilities.  This  test  requires  determining  a  figure  analogous  to 
an  unmanned  aerial  vehicle’s  (UAV)  position  relative  to  a  tar¬ 
get.  The  3-  and  5-digit  listening  tests  measure  selective  atten¬ 
tion  abilities.  This  test  requires  responding  when  any  of  the 
3  or  5  specified  numbers  are  presented  aurally.  The  horizon¬ 
tal  and  the  airplane  tracking  tests  measure  coordination.  The 
horizontal  tracking  test  requires  using  a  throttle  or  rudder  ped¬ 
als  and  the  aiiplane  tracking  test  requires  using  the  joystick  to 
track  a  moving  target.  Emergency  scenario  test  measures  deci¬ 
sion  making  under  stress  and  it  is  administered  along  with  the 
horizontal  and  the  airplane  tracking  test.  In  this  test,  the  par¬ 
ticipants  are  presented  with  three  emergency  scenarios  (one 
at  a  time)  that  must  be  canceled  out  by  typing  a  preinstructed 
three  letter  code  while  performing  the  horizontal  and  the  air¬ 
plane  tracking  tests.  Table  I  contains  a  complete  list  of  the 
TBAS  subtests. 

Procedure 

Participants  were  first  asked  to  read  and  sign  an  informed  con¬ 
sent  form  approved  by  an  institutional  review  board  (IRB). 
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TABLE  I.  Construction  of  the  TEAS  Tests 


Single  Tasks 

DOT:  Directional  Orientation  Test 
DL3:  3-Digit  Dichotic  Listening  Test 
DL5:  5-Digit  Dichotic  Listening  Test 

HTT:  Horizontal  Tracking  Test  (requires  throttle  or  rudder  pedals) 
ATT:  Airplane  Tracking  Test  (requires  joystick) 

Multitasks 

AHT:  Horizontal  Tracking  Test  -H  Airplane  Tracking  Test 
AHDL3:  Horizontal  Tracking  Test  +  Airplane  Tracking  Test  +  3-Digit 
Listening  Test 

AHDL5:  Horizontal  Tracking  Test  +  Airplane  Tracking  Test  +  5-Digit 
Listening  Test 

AHSC:  Horizontal  Tracking  Test  -I-  Airplane  Tracking  Test  + 
Emergency  Scenario  Test 


They  were  then  administered  the  TEAS  twice,  once  with 
the  throttle  and  once  with  the  rudder  pedals  with  an  interval 
of  2  hours  in  between  testing.  The  order  of  the  input  device 
was  counterbalanced  with  21  (49%)  participants  taking  the 
TEAS  first  with  the  throttle  and  22  (5 1  %)  participants  taking 
the  TEAS  first  with  the  rudder  pedals.  The  order  of  the  input 
device  for  each  subject  was  predetermined  by  using  a  random 
number  generator. 

The  participants  skipped  the  first  subtest,  the  direction 
orientation  test,  since  that  does  not  require  using  a  tracking 
device.  The  individual  listening  tests  were  included  in  the 
procedure  because  although  they  do  not  require  any  track¬ 
ing  device,  they  are  included  in  the  subsequent  multitasks  that 
require  tracking.  Therefore,  the  participants  started  the  TEAS 
with  the  3-digit-listening  test. 

The  participants  were  allowed  to  adjust  their  joysticks, 
throttle,  and  foot  pedals  to  feel  comfortable.  However,  the 
position  of  the  monitor  remained  stable.  The  keyboard  was 
positioned  at  the  center  of  the  units,  in  front  of  the  monitor; 
the  joystick  was  located  on  the  right  side  of  the  key  board; 
the  throttle  was  on  the  left  side  of  the  key  board;  and  the  foot 
pedals  were  approximately  directly  below  the  key  board.  The 
volume  of  the  headphones  was  adjustable  and  the  participants 
were  free  to  adjust  the  volume. 

RESULTS  AND  DISCUSSION 

At  the  time  of  the  analysis  only  the  individual  TEAS  subtest 
scores  were  available  and  each  of  these  subtests  scores  were 
analyzed  for  the  study.  Although  all  the  subtests  were  ana¬ 
lyzed,  for  the  purpose  of  this  study,  we  were  mostly  inter¬ 
ested  in  the  subtests  that  contain  horizontal  tracking  test  and 
require  a  throttle  or  foot  pedals.  These  subtests  included  hori¬ 
zontal  tracking  test  (HTT),  airplane  +  horizontal  tracking  test 
(AHT),  airplane  -i-  horizontal  tracking  +  3-digit  listening  test 
(AHDL3),  airplane  +  horizontal  tracking  -i-  5-digit  listening 
test  (AHDL5),  and  airplane  -t-  horizontal  tracking  -i-  emer¬ 
gency  scenario  test  (AHSC).  A  key  for  abbreviations  used  in 
the  analyses  is  presented  in  Table  I. 


Analysis  of  the  Impact  of  the  Input  Device 

Participants  were  categorized  into  two  groups.  Group  1  took 
the  TEAS  with  the  throttle  first  and  group  2  took  the  TEAS 
with  the  foot  pedals  first.  To  determine  the  impact  of  the  spe¬ 
cific  input  device,  the  performance  score  on  each  of  the  TEAS 
subtests  was  compared  between  groups  1  and  2.  The  results 
demonstrated  no  significant  input  effect  for  any  of  the  TEAS 
subtests. 

Analysis  of  Practice  Effect 

All  the  TEAS  scores  from  the  first  test  and  the  retest  were 
compared  for  each  participant  to  measure  practice  effect. 
Significant  practice  effects  were  found  for  all  of  the  subtests 
except  for  DL3  and  DL5.  For  example,  scores  for  HTT,  ATT, 
AHT,  AHDL3,  AHDL5,  and  AHSC  for  all  of  the  participants 
were  significantly  better  when  taken  the  second  time  regard¬ 
less  of  the  input  device,  t  (42)  =  -2.8,  p  <  0.05,  t  (42)  =  -5.4, 
p  <  0.001,  t  (42)  =  -5.0,  p  <  0.001,  t  (42)  =  -5.0,  p  <  0.001, 
t  (42)  =  -2.9,  p  =  0.005,  and  t  (42)  =:  -3.1,  p  <  0.005,  respec¬ 
tively.  However,  the  scores  for  DL3  were  in  fact  lower  when 
taken  the  second  time,  which  was  marginally  significant, 
t  (42)  =  2.0,  p  <  0.05.  The  scores  for  DL5  were  at  the  same 
direction  as  DL3,  but  not  significant,  t  (42)  =  1.5,  p  >  0.05. 

Interestingly,  just  as  the  combined  group,  the  practice 
effects  for  group  1  participants  remained  strongly  significant 
for  HTT,  ATT,  AHT,  AHDL3,  AHDL5,  and  AHSC  except  for 
DL3  which  consequently  became  nonsignificant.  However, 
the  practice  effects  for  group  2  participants  were  compro¬ 
mised  for  HTT,  AHDL5,  and  AHSC,  t  (21)  =  -1.1,  p  >  0.05, 
t  (21)  =  -1.9,  p  >  0.05,  and  t  (21)  =  -1 .0,  p  >  0.05.  Practice 
effects  for  ATT,  AHT,  and  AHDL3  in  group  2  became  weaker 
but  remained  significant,  /  (21)  =  -2.4,  p  <  0.05,  f  (21)  =  -2.7, 
p  <  0.05,  and  t  (21)  =  -2.4,  p  <  0.05,  respectively.  In  other 
words,  the  practice  effects  evident  in  the  combined  group  were 
mostly  due  to  group  1  participants  who  took  the  TEAS  with 
the  throttle  first.  This  finding  implies  that  it  is  easier  to  trans¬ 
fer  tracking  skills  learned  by  using  a  hand-controlled  throttle 
to  a  similar  task  using  foot-controlled  pedals.  However,  when 
the  initial  tracking  skills  are  acquired  by  using  foot  pedals  it  is 
less  transferable  to  a  similar  task  using  a  throttle. 

A  literature  search  in  this  area  revealed  no  relevant  study 
on  human  ability  to  transfer  learned  skills  from  hand  to  foot 
or  foot  to  hand.  However,  previous  research  has  demonstrated 
that  there  is  unique  structural  and  functional  specialization  that 
differentiates  human  upper  and  lower  limb  use.‘^  Fewer  neu¬ 
ral  resources  are  found  to  be  devoted  to  leg  and  foot  control 
than  to  arm  and  hand  control  and  we  usually  have  a  uniquely 
restricted  use  of  the  lower  limb.  In  addition,  the  status  of  leg 
and  foot  control  differs  from  that  of  arm  and  hand  control  both 
early  in  life  and  during  the  later  years  of  decline.'^  Therefore, 
it  is  not  surprising  that  a  hand-learned  skill  is  easier  to  transfer 
to  a  task  requiring  foot  control  than  a  foot-learned  skill  is  to 
transfer  to  a  task  requiring  hand  control. 

The  practice  effect  found  in  this  study  also  warrants  further 
exploration.  Although  in  this  study,  the  TEAS  was  readmin- 
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istered  with  different  input  devices  and  the  interval  between 
testing  was  only  2  hours,  the  demonstrated  practice  effect  was 
consistent  with  the  results  of  a  previous  study‘s  that  examined 
the  short-term  retest  reliability  of  the  BAT.  Carretta  found  a 
significant  practice  effect  when  Air  Force  pilot  training  candi¬ 
dates  took  the  BAT  twice  in  2  consecutive  days.  The  TBAS  and 
the  BAT  are  comparable  since  both  are  performance-based, 
computerized  tests  and  the  TBAS  is  the  revised  version  of 
the  BAT.  Out  of  the  BAT  subtests,  the  two-hand  coordination 
and  complex  coordination  tests  are  similar  to  the  TBAS  track¬ 
ing  tests  and  require  similar  input  devices  (e.g.,  control  stick 
and  rudder  pedals).  The  results  demonstrated  that  in  two-hand 
coordination  and  complex  coordination  tests,  tracking  error 
was  reduced  significantly  when  taken  the  second  time.  The 
author  of  that  study  concluded  that  additional  research  needs 
to  be  conducted  to  determine  the  test-retest  interval  required 
to  avoid  retest  improvement  due  to  prior  experience  with  the 
test  battery.  Consequently,  the  next  study  was  designed  to  fur¬ 
ther  investigate  the  effect  of  repeated  exposures  of  the  TBAS. 

Study  2 

The  purpose  of  study  2  was  to  further  explore  the  TBAS 
practice  effect  found  in  study  1.  This  study  was  designed  to 
administer  the  TBAS  twice  with  the  same  input  device  with  a 
24-hour  interval  between  testing. 

Method 

Participants 

Students  waiting  to  attend  aviation  preflight  indoctrination 
(API)  were  recruited  to  take  the  TBAS.  A  total  of  45  students 
participated  in  the  study.  However,  TBAS  records  for  sev¬ 
eral  students  were  either  incomplete  or  not  retrievable  and  a 
complete  set  of  36  TBAS  scores  were  used  for  the  analysis. 
Out  of  the  36  participants,  33  were  male  and  3  were  female. 
The  sample  of  participants  included  31  Caucasians,  3  Asians, 
1  Hispanic,  and  1  Native  American. 

Materials 

The  materials  were  the  same  as  in  the  first  study.  However,  the 
rudder  pedals  were  excluded  as  an  input  device  in  this  study. 

Procedure 

Similar  to  study  1,  all  participants  were  first  asked  to  read  and 
sign  an  informed  consent  form.  The  participants  were  admin¬ 
istered  the  TBAS  twice  in  2  consecutive  days  with  a  24-hour 
interval  in  between  testing.  Unlike  study  1,  the  participants 
were  tested  on  all  the  TBAS  subsets  including  the  direction 
orientation  test  (DOT;  see  Table  I).  All  the  TBAS  peripherals 
were  arranged  in  a  similar  manner  as  in  study  1 . 

RESULTS  AND  DISCUSSION 

Similar  to  study  I,  the  scores  from  each  of  the  TBAS  sub¬ 
tests  were  analyzed  for  this  study.  The  TBAS  is  designed  to 
record  both  correct  responses  and  time  duration  for  each  of 


the  subtests.  For  the  direction  orientation  test,  the  TBAS  gen¬ 
erates  scores  for  correct  responses,  elapsed  time  for  correct 
responses,  number  of  wrong  responses,  and  elapsed  time  for 
the  wrong  responses.  For  the  tracking  tests,  the  TBAS  incor¬ 
porates  both  correct  response  and  reaction  time  into  a  single 
score.  For  the  dichotic  listening  tests,  it  produces  the  number 
of  correct  responses  as  well  as  the  total  elapsed  time.  The  total 
numbers  of  correct  responses  as  well  the  total  elapsed  time 
were  taken  into  account  for  each  of  the  subtests  to  analyze  the 
practice  effect. 

For  the  single  tasks,  each  subtest  score  was  analyzed  for 
practice  effect.  For  the  multitasks  (see  Table  I),  a  single  score 
was  derived  by  adding  the  multiple  test  scores  similar  to  study 
1.  The  individual  test  scores  within  that  multitask  were  also 
analyzed  separately  for  practice  effect. 

Analysis  of  Practice  Effect  of  the  Single  Tasks 

All  the  TBAS  scores  from  the  initial  test  and  the  retest  were 
compared  for  each  participant  to  measure  practice  effect.  For 
the  single  tasks,  significant  practice  effect  was  noted  for  DOT, 
HTT,  and  ATT,  r  (35)  -  -4.4,  p  <  0.001,  t  (35)  =  -4.4,  p  < 
0.001  and  t  (35)  =  -6.3,  p  <  0.001,  respectively.  However, 
there  was  no  practice  effect  for  DL3  or  DL5,  t  (35)  =  -\.l,p> 
0.05  and  t  (35)  =  0.98,  p  >  0.05,  respectively. 

For  DOT,  total  elapsed  time  improved  for  correct  responses 
as  well  as  for  the  total  responses,  (35)  =  3.7,  p  <  0.01  and 
t  (35)  =  5.8,  p  <  0.001,  respectively.  For  the  dichotic  listening 
tasks,  higher  number  of  correct  responses  seemed  to  be  related 
to  larger  values  of  elapsed  time.  There  was  a  significant  pos¬ 
itive  correlation  between  DL3  score  for  correct  responses 
and  elapsed  time  as  well  as  between  DL5  scores  for  correct 
responses  and  elapsed  time,  r  =  0.91,  /?  <  0.01  and  r  =  0.53, 
p  <  0.01,  respectively.  There  was  only  a  minimal  variabil¬ 
ity  among  the  DL3  and  DL5  scores.  Twenty-six  participants 
responded  correctly  to  all  the  3-digit  stimuli  and  23  partici¬ 
pants  responded  correctly  to  all  of  the  5-digit  stimuli.  The  par¬ 
ticipants  with  perfect  scores  on  DL3  showed  no  significant 
difference  in  elapsed  time  on  the  second  trial,  t  (25)  =  -1.8, 
p  >  0.05.  However,  the  participants  with  perfect  scores 
on  DL5  had  significantly  higher  elapsed  time  on  the  retest, 
t  (22)  =  -3.5,  p  <  0.05.  In  other  words,  participants  with  all 
correct  responses  on  the  5-digit  listening  test  took  longer  to 
respond  on  the  second  trial. 

Analysis  of  Practice  Effect  of  the  Multitasks 

For  the  combined  test  of  AHT,  practice  effect  was  significant 
for  the  combined  test  score  as  well  as  for  the  airplane  and  hor¬ 
izontal  tracking  score  individually,  t  (35)  =  -7.3,  p  <  0.001, 
t  (35)  =  —7.5,/?  <  0.001  and  t  (35)  =  -3.4,/?  <  0.005,  respectively. 
For  AHDL3,  practice  effect  for  the  overall  score  as  well  as  the 
individual  test  scores  of  ATT  and  HTT  were  significant,  t  (35)  = 
-4.8,  p  <  0.001,  t  (35)  =  -7.0,  p  <  0.001  and  t  (35)  =  -4.3, 
p  <  0.001,  respectively.  However,  DL3  score  within  AHDL3 
was  not  significant  t  (35)  =  -0.07,  p  >  0.05.  For  AHDL5,  the 
total  score  as  well  as  individual  scores  on  ATT  and  HTT  were 
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significant,  t  (35)  =  -53,  p<  0.001  (35)  =  -5.1,p  <  0.001  and 
t  (35)  =  -4.3,  p  <  0.001,  respectively,  but  the  DL5  score  was 
not  significant,  t  (35)  =  -1.0,  p  >  0.05.  Similarly,  for  AHSC, 
the  total  score  as  well  as  individual  scores  on  ATT  and  SC 
(emergency  scenario  test)  were  significant,  t  (35)  =  -3.2,  p  < 
0.005,  t  (35)  =  -5.2,  p  <  0.001  and  t  (35)  =  -4.3,  p  <  0.001, 
respectively,  but  HTT  was  not  significant,  t  (35)  =  -1.3,  p  > 
0.05. 

Correlational  Analyses  of  Total  Flight  Hours  and 
Practice  Effect 

The  participants  had  a  range  of  0  to  400  total  hours  of  flight 
training  at  the  time  they  took  the  TEAS  (M  -  49;  SD  =  86). 
The  total  hours  of  flight  training  were  negatively  correlated 
with  practice  effect  on  DOT  and  AHT,  r  =  -0.40,  p  <  0.05  and 
r  =  -0.35,  p  <  0.05,  respectively.  In  other  words,  participants 
with  more  flight  experience  showed  less  improvement  in  per¬ 
formance  on  the  second  trials  of  DOT  and  AHT.  This  is  not 
unexpected  with  tests  that  are  correlated  with  flight  training 
performance  because  the  participants  with  more  flight  experi¬ 
ence  may  start  out  with  higher  scores  on  DOT  and  AHT  and 
are  therefore,  less  likely  to  improve  significantly  in  the  sub¬ 
sequent  trials.  However,  the  correlation  between  flight  hours 
and  performance  on  the  first  trials  of  DOT  and  AHT  fell  short 
of  being  significant  although  the  trend  was  in  the  expected 
direction. 

The  results  of  this  study  demonstrated  that  TEAS  perfor¬ 
mance  in  all  the  subtests  except  for  the  ones  including  dichotic 
listening  tests  improved  when  the  participants  took  the  test  the 
second  time.  The  practice  effect  found  in  this  study  is  con¬ 
sistent  with  the  results  of  study  1,  which  found  significant 
practice  effect  for  most  of  the  TEAS  subtests  with  a  2-hour 
interval  between  testing. 

GENERAL  DISCUSSION 

These  preliminary  studies  provided  the  directions  needed  to 
modify  the  TEAS  to  incorporate  it  into  the  naval  aviation 
selection  process  in  an  effort  to  reduce  attrition  and  training 
costs,  improve  pilot  performance,  and  enhance  aviation  opera¬ 
tional  safety  in  the  fleet.  The  current  studies  were  specifically 
designed  to  provide  insight  into  the  appropriate  input  devices 
and  the  adequate  test-retest  interval  for  the  performance- 
based  measures  of  the  redeveloped  ASTE. 

The  results  demonstrated  that  TEAS  performance  was  not 
affected  as  a  result  of  the  specific  input  device  used  for  the  hor¬ 
izontal  tracking  tests.  Thus,  although  the  original  TEAS  was 
designed  to  use  foot  pedals  for  the  horizontal  tacking  tests,  the 
foot  pedals  can  be  substituted  for  throttle  without  having  any 
significant  impact  on  overall  TEAS  performance. 

These  results  also  established  that  TEAS  performance  does 
improve  with  repeated  exposure.  However,  in  this  study  the 


retest  was  administered  after  only  24  hours.  Therefore,  follow¬ 
up  studies  should  further  investigate  the  relationship  between 
practice  effect  and  the  test-retest  interval.  It  would  be  useful  to 
test  the  TEAS  practice  effect  according  to  the  ASTE  retest  pol¬ 
icy.  Currently,  pilot  candidates  are  allowed  to  take  the  ASTE 
3  times  with  30  days  gap  between  tests  1  and  2  and  60  days 
gap  between  tests  2  and  3.  Therefore,  TEAS  practice  effect 
should  be  examined  after  30  days  and  again  after  60  days  to 
determine  whether  candidates  should  be  allowed  to  retake  the 
TEAS  according  to  the  authorized  ASTE  retake  interval. 
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